Application of Convolutional Neural Network for Classification of Rice Plant Disease¶

STA141A Final Report¶

Georgiy Kiselev 919184826¶

June 12, 2023¶

1 Abstract¶

Accurate diagnosis of rice disease is crucial for correct treatment. However, rural rice-growing regions in India and southeast Asia often rely on individuals specialized in the growing of rice. Such a dependency is inefficient, as the avaiability of any such person is highly limited, especially if it is necessary to examine vast paddies of rice. A misdiagnosis or an unexamined plant can prove costly in the long run, especially if the diseased plant begins infecting its neighbors, causing large drops in yield. This paper explores the application of machine learning processes for the automatic detection and diagnosis of rice plant diseases. The model was trained, validated, and tested on a dataset of approximately 10,000 images of 10 classes, with one class representing healthy rice plants, and the other 9 representing 4 main archetypes of disease: bacterial, fungal, viral, and insect-caused. More specifically, the diseases include the top 9 most common rice diseases: leaf blight, leaf streak, panicle blight, blast, hispa, tungro, dead heart, brown spot, and downy mildew. The diagnosis method utilized the MobileNetV2 architecture, due to its computational efficiency and lightweight structure. This model was able to achieve 88.31% accuracy on the test set of images. This is a satisfactory performance, due especially to the fact that some diseases appear similar.

2 Introduction¶

Rice is one of the most important food energy sources in the world. It is a complex carbohydrate that contains multitudes of beneficial micronutrients, including B vitamins, potassium, folic acid, fiber, iron, zinc and magnesium [1]. More than half of the world's population depends on rice daily as a staple food. As a result, it is grown in more than 100 countries, with approximately 90% of the total worldwide production coming from Asia [2]. However, the amount of rice produced by these countries is not completely efficient. Due to numerous diseases - bacterial, viral, or other, yields in rice-growing regions are greatly reduced. Certain rice diseases, can end in the loss of up to 80% of a crop! [3]. Often, the symptoms of rice disease appear very similar, and it may be difficult to accurately diagnose the specific disease without use of specialized equipment [4]. As such, it would be practical to employ machine learning concepts for the construction of a neural network capable of automatically detecting and diagnosing disease in rice plants.

The data used for this paper is taken from the open source 'Paddy Doctor' repository online, where the original dataset is quite a bit larger than what is used for training the model here. The images of healthy and diseased rice plants on the 'Paddy Doctor' website are taken from Pallamadai Village in Tamil Nadu, India. The smaller dataset used in this paper has 9 diseases, all of which are not unique to Southern India [5]. This allows for the model to be applicable to other rice-growing regions, such as Japan, Southeast Asia, Australia and Europe [6].

The idea of applying convolutional neural networks to the problem of disease classification is not a new concept by any means [7], other research focuses on larger, more thorough models such as ResNet50, ResNet121 and Vgg-16 and 19.

Overall, the application of machine learning to the diagnosis of rice diseases can provide a much needed boost in the efficacy of treatment and early prevention of diseases that would otherwise cause massive loss in yield volume.

However, there are three specific problems that may affect the capacity of the neural network in correctly diagnosing diseases. The first is the concern that symptoms of some diseases in the dataset appear very similar to each other, despite having distinct effects and treatments. The second is that the images are not segmented, so the diseased subjects often blends in with the background and it is hard to distinguish where the disease is. The third is that the images are sometimes blurry - the camera focuses on the background, rather than the diseased subject in the foreground.

2.1 Convolutional Network¶

This new age approach to image classification has revolutionized the process of diagnosis. Convolutional Neural Networks (CNNs) are able to recognize global patterns in images by continually learning small local patterns and combining them. As a result, CNNs are less susceptible to the exploding gradient issue seen in FCCs (Fully Connected Networks). Further, CNNs are translation invariant, meaning that once they learn a pattern appearing in an image, they will recognize it regardless of where it appears on subsequent images. FCCs must relearn the pattern for every new location, and are thus less efficient than CNNs.

The architecture of a CNN consists of several important layers briefly discussed below:

Convolutional Layer: The tensor of shape (input height) x (input width) x (input depth). In this case, the depth is color channels in the image (3) representing RGB values. The layer then slides a small kernel (filter window) across the image and captures numerical values associated with each pixel. Due to not being fully connected, the number of parameters is reduced and the network can be used more efficiently on larger images.

Pooling Layer: The Max Pooling layer seen in the architecture of later sections is a downsampling layer that selects the largest values from each convolution window as the inputs to the next layer. Thus, the pooling layer serves to reduce dimensionality from each neuron cluster before proceeding to the next neuron.

Flattening layer: The Flattening layer compresses the previously computed feature maps into a 1 dimensional array in order to match the input shape of the following fully connected layer.

Fully Connected layer: This Neuron connects all neurons of one layer to all neurons of a following layer. This is the output layer of the convolutional Neural Network which will classify the images.

3 Exploratory Data Analysis¶

By displaying the top 5 results of our CSV, we see that the data has four variables.

image_id: The image index associated with the image folder.

label: The type of disease the rice is affected with.

variety: The name of the type of paddy.

age: Age of the paddy.

There are 10,407 observations in the dataset, each with the above 4 variables.

Age is the only non-categorical variable. We can see that the average paddy age for this dataset is about 64 days. This suggests that many paddies are either young, or just reaching maturity [1]. Indeed, we see that the maximum paddy age feature in the data is just 82 days.

From the plot, ADT45 is the most prevalent type of paddy in the data. Due to this, it would be pertinent to further examine its statistics.

As expected, the amount of ADT45 paddies included greatly influences the summary statistics of the entire dataset. It includes both the minimum and maximum paddy ages (45 and 82 days, respectively). The mean age of the ADT45 paddies also reflects the overall mean at 64.3 days.

Now, examine the types of diseases included.

There are 10 classes, with 9 of them being diseases and one being healthy rice, represented by the 'normal' label.

Across the disease archetypes the classes are more balanced. However, there is a predominance of insect and fungal diseases. This makes sense due to the fact that the warm, humid conditions of rice growing environments are also well-suited for insects. The humidity also provides excellent moisture for the growth of fungi, which explains why it is the second most prominent category.

Below is a collection of labeled images (one from each class in the data) to demonstrate the prominent symptoms of each disease. (These images were handpicked from the data in order to best demonstrate the symptoms. In later sections, images will be randomly selected when data challenges are discussed).

3.1 Data Challenges¶

Based on these images, the challenges discussed previously become apparent. In this section, I will examine the extent to which each challenge affects the data.

3.1.1 Similarities Between Diseases¶

Several of the diseases in the dataset show similarities to one another. One such pair is Dead Heart and Bacterial Pannicle Blight. Despite being caused by completely different things (insect vs bacterium, respectively), they present with the same symptom - a withered, brittle panicle. (The panicle of rice is the flowering part where the grain grows). For a base examination, I randomly sample 8 images from the training data (4 of each disease) to demonstrate the similarities in appearance.

As apparent, to the untrained eye, it may be virtually impossible to distinguish between riceplants affected by these two diseases. The hope is that the algorithm will be able to pick up on differences much more efficiently and accurately than a human would.

3.1.2 Diseased Subject Blending in with Background¶

As a result of bad photography or minimal symptoms, some diseased subjects blend in with the background to demonstrate, I will randomly sample images from the training dataset. I will not label these image plots, since labels are not relevant to the segmentation concern.

Specifically with the diseases that affect the leaf, it is very difficult to discern the primary subject without segmentation. The hope is that the algorithm will not become lost in the background while searching for local patterns. Furthermore, some of the images in the random sample show such minimal effects of the disease that they almost appear healthy, which can be very misleading. Incorrectly diagnosing minimal symptoms can lead to huge yield loss later on. I further hope that by keeping a constant 3x3 windows for each convolutional layer, the small appearance variations between healthy and diseased can be picked up on by the network.

Examine a subset of the above images, which are virtually undiscernable due to a combination of challenges mentioned perviously.

For instance, it is unclear what the subject of the second image is, since there is no leaf being held up to the camera. Furthermore, the foreground of the images is blurry, so any symptoms of disease are not immediately apparent. Thirdly, The greenness of the pictures overwhelm the image, and make it diffuclt to process what is being shown.

4.1 Benchmark MobileNetV2 Model¶

To demonstrate benchmark effectiveness, I train a default MobileNetV2 model for 10 epochs.

4.1.1 Augmenting and Preprocessing image Data¶

First, I resize and apply certain transformations to the dataset. These transformations augment the dataset by treating the same image as several new ones that appear as completely unique to the model. The transformations include: zoom, shear, rotation, gaussian blur, X and Y shifts, and vertical/horizontal flips. (These transformations will only be applied to the training, it is important to leave testing/validation data unchanged).

Then, pass the transformed images through a two data generators, training and validation, in order to vectorize them for use in the model. We check that the generators worked by outputting how many images were found and in how many classes. For splitting, I use proportions of 80/10/10. (80% of the images are for training, 10% for validation, and 10% for testing). It is important to include the largest part for training so that the model may accurately learn the abstract features of the dataset.

This is the desired result. from our overall dataset of 10,407 images, 8,323 are used for training and 1,036 are used for validation.

This model performs well, but not up to standards with the predictive power we are attempting to achieve. The discrepancy between training and valdiation accuracy is a common trait of overfitting, which means our model may be optimizing to the training data instead of generalizing to new data. In the next section I increase the number of trainable parameters un-freezing some of the top layers in the MobileNetV2 architecture and allowing them to train. Due to having an expanded model, I will also train it for more epochs.

Also, the curves for validation accuracy are jagged, rather than smooth, which is further evidence of overfitting.

4.2 Partially Unfrozen MobileNetV2 Model¶

I will unfreeze the last 30 layers of the MobileNetV2 network, which will allow the last convolutional block to train new weights based on the image data we are providing. Furthermore, I will increase training time from 10 to 30 epochs. This will allow the now unfrozen layers to spend more time learning the spatial hierarchy of the dataset.

As apparent from these metrics, the unfrozen model performs much better than the initial MobileNetV2 model, reaching a top validation accuracy of 88.13%. It thus follows that un-freezing the last 30 layers of the MobileNetV2 architecture allows it to excellently generalize to the data, preventing the overfitting experienced with the initial model. Furthermore, it appears that training the model for more than 30 epochs would generally continue to increase the performance accuracy.

Furthermore, both loss and accuracy curves are fairly smooth, which is a good indicator that the model is NOT overfitting to the training data. This is a good sign, because our model is making the generalization versus optimization tradeoff. By not overfitting to the trading data (optimization) it is able to better perform on external data (generalization), and thus produces better results when applied to data we care about.

5 Predictive Performance¶

The test dataset for this paper consists of 1048 images, once again distributed between 10 classes (9 diseased 1 healthy). However, the model will not be shown the true labels of this dataset, and will instead be asked to make predictions based on the abstract features it has learned during training.

From the validation accuracy found earlier, I predict that the test accuracy should be somewhere in the range of 85-90%.

Using the baseline 'evaluate' command included in the Keras API, we see that the model performs with 87.31% accuracy on the test dataset. This is significant because it performs well on data it has not seen before, signifying that it did not overfit during training. Next, we will use the 'predict' function to see what exactly the model loses 12.69% accuracy on. Hopefully, we can get insight on how the aforementioned data challenges affect the model's performance.

As we can see, the prediction accuracy of the model is the same, which means it is acceptable to use the newly made dataframe containing true labels and predictions in the construction of a confusion matrix.

This confusion matrix tells us a lot of things about the performance of the unfrozen model and how the algorithm responds to the data challenges mentioned earlier.

To begin: We can see that the class-wide (rows) error rates range from 5% to 30%. The classes with more representation (i.e. more images) have much lower error rates. This makes sense because the disease distribution in the test data follows the distribution across the training data, and the model is able to better learn the features for diseases of which it has more samples.

We can provide concrete examples of the discrepancies based on class sizes by examining the maximum and minimum of the error range. The normal (healthy) class has 177 images in the testing data (17% of the data). Consequently, the model misclassifies healthy rice at a rate of 5%. On the other hand, the class of bacterial leaf blight only contains 45 images in the test set (4.3% of the data). As expected, it has a much larger (30%) misclassification rate. This result is strange, as the model should automatically adjust weights based for imbalanced classes.

Furthermore, we can learn more about the model by examining the columns of the confusion matrix. From the column labeled 'normal' we can see that our model has a False Negative rate of 16.4%. This is serious, because labeling diseased rice as normal is potentially more dangerous than misclassifying one disease for another. Such an error is most likely caused by the aforementioned data challenges.

Based on the columns of the matrix, we can also see that misclassification among diseases seems correlated to prominence of symptoms. For example, dead heart is the least misclassified, most likely due to the withered pannicle being a feature much different than the leaf streaks or spots common for the other classes. Proportionally, Panicle Blight also has relatively few misclassifications, since it is also a disease affecting the rice panicle.

Utilizing the image indices of the misclassified samples, we can examine some of the images that confused the algorithm.

Each of these images was incorrectly classified as a healthy plant. It is also clear that each image suffers from a Data Challenge discussed in section 3. It is unclear where the diseased subject is, with green (presumably healthy) rice taking up most of the frame. In the case of the rightmost image (dead heart) the panicle is completely gone, so despite the model having learned the main features of dead heart, it is unable to find it within the image.

Despite challenges with the data, the algorithm still performed very well in correctly labeling the majority of the diseases. This implies that the convolutional network automatically performs segmentation and is able to somewhat disregard the background when assessing diseased subjects. Furthermore, the blurry images did not seem to pose much of an issue, as the algorithm picked up on important pixel values and formed a working spatial hierarchy regardless of the resolution.

It would also be pertinent to examine correctly classified images. This will demonstrate how despite similaritiesin appearance, the network learned unique local patterns for each class.

I will refer to these images as 'Image 1', 'Image 2', 'Image 3' and 'Image 4,' with the leftmost being Image 1 and the rightmost Image 4. (As a side note, I disabled the greyscale transformation because color values seemed relevant to the classification problem).

I will discuss how the algorithm overcomes the Data Challenges for each image and makes a correct prediction based on visible patterns.

Image 1: Is the image of a correctly classified case of Downy Mildew. The primary feature recognized by the model is the withered, twisted leaf that is a common symptom of late-stage Downy Mildew. Despite the diseased subject being out of focus, the algorithm recognizes the deviations in color gradient (from green to brown) and marks it as a region of interest.

Image 2: Is the image of a correctly classified case of Hispa. The primary features recognized by the model are the long white streaks parallel to the leaf's midrib. In addition to this, the beetle causing these symptoms is present on the leaf, but is most likely not an important feature learned by the network.

Image 3: Is the image of a correctly classified case of Tungro. The primary feature recognized by the model is the bright yellow-orange color the affected leaf becomes. Despite the primary subject being out of focus, the algorithm is once again able to recognize that the color deviations (from green to yellow-orange) are significant.

Image 4: Is the image of a correctly classified case of Dead Heart. The primary feature recognized by the model is the grey rice panicle (should be green). However, the model was also able to navigate around one of the Data Challenges, which mentioned the similarity between Dead Heart and Bacterial Panicle Blight. This happens due to the fact that with Dead Heart, the entire panicle withers and dies. With Panicle Blight, the affected panicle withers in most areas, but remains green in some areas. As such, the model was able to learn meaningful differences between the two diseases.

6 Discussion and Conclusion¶

As becomes apparent from the previous sections, the convolutional neural network approach to the diagnosis and classification of disease in rice plants proved to be quite effective.

With a final test accuracy of 87.31% the model demonstrated its ability to quickly AND accurately diagnose diseases. While a human expert may be able to more accurately diagnose rice illness, this is time-consuming, and experts in such a field are extremely limited. Meanwhile, the model constructed and trained earlier is able to make predictions instantaneously and could be widely deployed without issue. Interestingly enough, when compared to formal research, the model created here does not perform much worse! For example, the ensemble model created by (Deng, Tao, et al., 2021) reached an overall accuracy of 91% in diagnosing 6 types of diseases [8].

In regards to the challenges put forth during exploratory data analysis: The convolutional network was able to handle the challenges automatically, without significantly affecting its performance. This implies that the algorithm either does not take the challenges into account, like humans do, or simply resolves it by itself during data preprocessing.

Comments on increasing accuracy: If more time and computing resources were available, the performance of the model could be further boosted. That is, training for more epochs would most definitely increase test accuracy, since my model was only trained on 30 epochs and validation/training accuracy never decreased. Furthermore, usage of larger images may or may not increase performance. Generally, CNNs perform best on images of up to 256x256 depending on the architecture. This model was trained on 224x224, the standard image size MobileNetV2 expects. Of course, there is also room to near endlessly tune every hyperparameter value to marginally increase performance.

To conclude: the convolutional neural network approach has proved itself to be a powerful, independent tool for image classification, avoiding the many pitfalls of old densely connected networks. It does not rely on heavily preprocessed image data (needing only image resizing and pixel value normalization). Furthermore, the model created for the purpose of this paper is rather lightweight, being only 250 megabytes when compressed as a TFLite h5 weight file. This small size makes it perfect for mobile deployment, further increasing its accessibility.

Appendix¶

Importing Data on Colab¶

EDA sections¶

Data Challenges Discussion Code¶

Initial Model Code¶

Plotting Model Metrics¶

Training Unfrozen Model¶

Unfrozen Model Metrics¶

Predictive Performance¶

Acknowledgements¶

1. Rice Nutrition Information
2. Rice Importance for Global Nutrition
3. Rice Diseases
4. Basal Rice information including common diseases
5. Rice Diseases of India
6. Geographic Rice Sectors
7. Rice Disease Diagnosis with ML
8. Ding, Tao, et al., 2021